Skip to content

feat: extract build_gateway_client into packages/llm (#93) - #321

Merged
ckrough merged 2 commits into
mainfrom
issue/93
Aug 5, 2026
Merged

feat: extract build_gateway_client into packages/llm (#93)#321
ckrough merged 2 commits into
mainfrom
issue/93

Conversation

@ckrough

@ckrough ckrough commented Jul 27, 2026

Copy link
Copy Markdown
Member

Executes the ADR 0028 follow-up: promotes build_gateway_client out of retriever into a new shared package so petdata and biowriter can import one helper rather than copying it.

Closes #93

What changed

  • New packages/llm (dist evermore-llm, import evermore_llm), mirroring packages/auth: hatchling, py.typed, Apache-2.0 SPDX headers, same ruff/mypy/pytest config.
  • build_gateway_client moves there with its body unchanged. A structural GatewaySettings Protocol is the settings seam, so the package imports nothing from any service (ADR 0001). retriever.config.Settings satisfies it structurally with no runtime change.
  • retriever consumes it via a path-editable [tool.uv.sources] entry and drops its own copy. GatewayScope now has a single definition, in the package.
  • Test split: 6 pure-builder tests move to packages/llm/tests, driven by a Protocol-conforming stub; the 3 that need the real retriever.config.Settings stay in retriever (moving them would import a service into a package).
  • retriever.config imports GatewayScope under TYPE_CHECKING, keeping the openai SDK out of the settings import chain. import retriever.config no longer loads openai.

Pipeline phases

Phase Notes
🟡 Plan / design triage Adopted the worktree plan (provenance validated: base_sha ancestral, body digest matches). Design pass skipped, complexity score 2.
🟢 Implement Extraction applied; moved tests keep their original assertions verbatim.
🟢 Test-validation 0 blocking, 2 advisory.
🟢 Review-gate APPROVE (1 cycle), plus a delta re-review APPROVE after simplify.
🟡 Simplify Applied 1 change (the TYPE_CHECKING import move); delta re-reviewed APPROVE.
🟢 Gates test + lint both exit 0.
🟡 Security review 0 blocking, 3 advisory (accepted risk, see below).
🟢 PR opened Non-draft, referencing the issue.

Test results

Gate Command Result
test (cd services/retriever && uv sync && uv run python -m pytest tests/ --ignore=tests/integration) && (cd apps/stacker && npm ci && npm run build) exit 0: 379 passed, 10 deselected, coverage 89.43% (floor 80); stacker build clean
lint (cd services/retriever && ruff check + ruff format --check + mypy --strict) && (cd services/petdata && same + bandit) && (cd apps/stacker && npm run check) exit 0: retriever 67 files clean, petdata 25 files clean, bandit clean, svelte-check 4414 files, 0 errors, 0 warnings
new package cd packages/llm && ruff check + ruff format --check + mypy src/ + pytest exit 0: 6 passed

Eval scores

No evals configured in this repo.

Verification beyond the gates

Mutation replay against the split suite (scratch copy; worktree restored and checksum-verified):

Mutation Caught by
default_headers set unconditionally ..._omits_auth_header_when_token_absent
api_key=token (placeholder dropped) 3 tests
auth header name hardcoded ..._uses_configured_header_name
scope is None inverted retriever's retained ..._uses_scoped_chat_token

Both branches of the token logic stay covered across the split.

On acceptance criterion 3

"petdata and biowriter import the same helper" is satisfied by construction, not as current fact: petdata has no openai dependency and biowriter is not scaffolded, so neither consumes the gateway yet. What this PR delivers is a single shared home with no copy and no cross-service-import path remaining. The ADR and package README state this as forward-looking rather than claiming consumers that do not exist.

Accepted risks and deferrals

  • Advisory (medium): the SSRF invariant moved from the type system to call-site discipline. GatewaySettings is structural, so any duck-typed object now type-checks where retriever.config.Settings was previously required, and llm_gateway_base_url reaches AsyncOpenAI(base_url=...) with no scheme or host check while the real bearer is attached. Runtime enforcement is unchanged (the old annotation was TYPE_CHECKING-only too, so nothing validated at runtime before either). Not reachable today: retriever's call sites pass get_settings(). Accepted for this no-behavior-change extraction; adding a scheme or allowlist guard in the builder is a design change worth its own issue.
  • Advisory (low): the 6 moved package tests run in no CI job. packages/** fans out to the retriever and petdata jobs, but those run working-directory: services/<svc> and invoke only that service's pytest. This matches the existing gap for packages/auth and packages/schema; a standalone packages CI job was deliberately deferred rather than adding a one-package job here.
  • Advisory (low): packages/llm/uv.lock resolves openai 2.48 while retriever runs 2.26. The package's credential-placement assertions test against SDK internals at a version production does not run.
  • Advisory (low): docs/security/coverage-ledger.md line 183 asserts its row set equals git ls-files apps/stacker packages exactly. That count invariant is now stale, and 6 of the 7 new packages/llm/ files have no ledger row. This PR only re-pointed the one existing row's path.
  • Deferred by the plan: no standalone packages/** CI job, no pydantic settings mixin, no move of gateway_token_for / llm_gateway_base_url into the package.

ckrough added 2 commits July 27, 2026 13:34
Promote the gateway-configured AsyncOpenAI client builder out of retriever
into a new shared package, executing the ADR 0028 follow-up.

- New packages/llm (dist evermore-llm, import evermore_llm), mirroring
  packages/auth: hatchling, py.typed, Apache-2.0 headers, ruff/mypy/pytest.
- build_gateway_client moves there with its body unchanged. A structural
  GatewaySettings Protocol is the settings seam, so the package imports
  nothing from any service (ADR 0001).
- retriever consumes it via a path-editable [tool.uv.sources] entry and
  drops its own copy. GatewayScope now has one definition, in the package.
- Tests split: 6 pure-builder tests move to packages/llm/tests driven by a
  Protocol stub; the 3 that need the real retriever.config.Settings stay.
- retriever.config imports GatewayScope under TYPE_CHECKING, keeping the
  openai SDK out of the settings import chain.

Gates: test and lint both exit 0. retriever 379 passed at 89.43% coverage,
packages/llm 6 passed, stacker build and svelte-check clean, bandit clean.
The retriever Dockerfile builds from the repo root and COPYs each
packages/* path source named in [tool.uv.sources]. Adding evermore-llm
as a path source without a matching COPY left /app/packages/llm absent,
so uv sync --frozen failed with 'Distribution not found at:
file:///app/packages/llm' and the container-build (retriever) CI job
exited 2.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create packages/llm (shared gateway client)

1 participant